test: reduce flaky tests #1266
Merged
test: reduce flaky tests #1266
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
benjlevesque
force-pushed
the
test/reduce-flaky-tests
branch
from
29bbd60
to
76bb171
Compare
benjlevesque
commented
Nov 27, 2023
Comment on lines
-95
to
-118
| describe('Superfluid framework', () => { | ||
| it.each([ | ||
| { network: 'goerli' }, | ||
| { network: 'matic' }, | ||
| // { network: 'xdai' }, | ||
| { network: 'optimism' }, | ||
| { network: 'avalanche' }, | ||
| { network: 'arbitrum-one' }, | ||
| ] as Array<{ network: CurrencyTypes.EvmChainName }>)( | ||
| 'Should initialize superfluid framework on $network', | ||
| async ({ network }) => { | ||
| const provider = getDefaultProvider(network); | ||
| const networkValidRequest = { | ||
| ...validRequest, | ||
| currencyInfo: { | ||
| ...validRequest.currencyInfo, | ||
| network, | ||
| }, | ||
| }; | ||
| const sf = await getSuperFluidFramework(networkValidRequest, provider); | ||
| expect(sf).toBeDefined(); | ||
| }, | ||
| ); | ||
| }); |
There was a problem hiding this comment.
This is definitely not a "unit" test. Considering the flakiness it creates, I'm dropping it and not replacing it.
benjlevesque
commented
Nov 27, 2023
Comment on lines
-56
to
-80
| describeIf(!process.env.CIRCLE_PR_NUMBER && !process.env.DISABLE_API_TESTS, 'Multichain', () => { | ||
| // TODO temporary disable xDAI, CELO, Sokol, and Goerli | ||
| // FIXME: API-based checks should run nightly and be mocked for CI | ||
| [ | ||
| 'mainnet', | ||
| // 'rinkeby', | ||
| // 'goerli', | ||
| // 'xdai', | ||
| // 'sokol', | ||
| 'fuse', | ||
| //'celo', | ||
| const infoRetriever = new EthInputDataInfoRetriever( | ||
| paymentAddress, | ||
| PaymentTypes.EVENTS_NAMES.PAYMENT, | ||
| 'matic', | ||
| 'fantom', | ||
| ].forEach((network) => { | ||
| it(`Can get the balance on ${network}`, async () => { | ||
| const retriever = new EthInputDataInfoRetriever( | ||
| '0xc12F17Da12cd01a9CDBB216949BA0b41A6Ffc4EB', | ||
| PaymentTypes.EVENTS_NAMES.PAYMENT, | ||
| network, | ||
| '9649a1a4dd5854ed', | ||
| process.env[`EXPLORER_API_KEY_${network.toUpperCase()}`], | ||
| ); | ||
| await expect(retriever.getTransferEvents()).resolves.not.toThrow(); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
removed this test. This is covered by the first test in terms of logic. The rest isn't "unit" test, but integration, and is too flaky. This is an obsolete feature and doesn't deserve the effort imo
github-actions
bot
requested review from
KolevDarko,
leoslr,
MantisClone and
yomarion
KolevDarko
approved these changes
Nov 28, 2023
benjlevesque
requested review from
alexandre-abrioux
and removed request for
yomarion
alexandre-abrioux
approved these changes
Nov 29, 2023
benjlevesque
force-pushed
the
test/reduce-flaky-tests
branch
from
d30f9c0
to
d27db26
Compare
leoslr
approved these changes
Nov 29, 2023
MantisClone
approved these changes
Dec 12, 2023
| ...args: [string | number | Function | jest.FunctionLike, jest.EmptyFunction] | ||
| ) => (condition ? describe(...args) : describe.skip(...args)); | ||
| it('can detect a MATIC payment to self', async () => { | ||
| // NB: The from and to are the same |
There was a problem hiding this comment.
It might just be me but I always get tripped up by the NB acronym for Nota Bene. I think "Note" would be more clear.
Suggested change
| // NB: The from and to are the same | |
| // Note: The from and to are the same |
Co-authored-by: Alexandre ABRIOUX <alexandre-abrioux@users.noreply.github.com>
benjlevesque
force-pushed
the
test/reduce-flaky-tests
branch
from
9dec3a3
to
a2642ac
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #1092
Based on CircleCI insights
Remove unnecessary flaky tests
Replace network calls by mocks
Increase timeouts
Replaced BTC by USD if relevant (for the record, BTC was the only supported currency at first, but now it's legacy and uses an inefficient balance detector)